Similarity Measures for Smooth Web Page Classification

This thesis examines the application of consistency learning techniques for the classification of hyperlinked web pages. Different data similarity measures between the web pages are defined, using local features like textual content and features of the linked pages as a graph. The pairwise object similarities are gathered in similarity matrices, each of which can be used together with methods from consistency learning to make classification smooth with respect to the data structure revealed by these similarity matrices and improve the accuracy of a simple text classifier. To achieve even better performance in the primary task of web page classification, a secondary machine learning problem is defined as finding the optimal similarity matrix combination. The results on several hyperlinked text collections, including the well known WebKB collection show significantly better accuracy of the smooth learning methods over the plain text classification. The main novel contribution of this thesis is the definition and testing of various similarity measures between web pages and the construction of a locally flexible similarity measure from heterogeneous data sources that improves classification accuracy on each of them. These ideas may also be used with little modification in other domains besides web page classification, like bioinformatics and citation graph classification.